Understanding Drawings by Compositional Analogy

We describe an analogical method for constructing a structural model from an unlabelled 2D line drawing. The source case is represented as a schema that contains its 2D line drawing, the lines and intersections in the drawing, the shapes in drawing, and the structural model of the device depicted in the drawing. Given a target drawing and a relevant source case, our method first constructs a graphical representation of the lines and the intersections in the target drawing, then uses the mappings at the level of line intersections to transfer the shape representations from the source case to the target, next uses the mappings at the level of shapes to transfer the structural model of the device from the source to the target. The Archytas system implements and evaluates this method of compositional analogy. 1 Motivation and Goals We view the task of interpreting drawings as one of constructing a model of what the drawing depicts; the model enables higher-level (i.e. non-visual) inferences regarding the depicted content of the drawing. For example, in the context of CAD environments, the input to the task may be an unannotated 2-D vector graphics line drawing depicting a kinematics device, and the output may be a structural model of the device, i.e. a specification of the configuration of the components and connections in the device. Current methods [Ferguson and Forbus, 2000; Alvarado and Davis, 2001] for extracting a model from a drawing rely on domain-specific rules. In this paper, we propose to derive a model by analogy to a similar drawing whose model is known. This requires (1) an analogical mapping from the source (known) to the target (input) drawing on the basis of shapes and spatial relations, and (2) transfer and adaptation of the model of the source drawing to the target. Structure-mapping theory [Falkenhainer et al., 1990] views analogy-based comprehension as a process of mapping individual relations from the source to the target. Candidate inferences about these mappings are guided by higher-order relations such as causal relations. While this works well in certain domains, there are no clear higher-order relations in the geometric and spatial information explicitly encoded in a a